feat: chain orchestrator #185

frisitano · 2025-07-02T19:06:49Z

closes: #182

Copilot

Pull Request Overview

This PR implements a new ChainOrchestrator to replace the previous indexer, integrates it throughout the node, watcher, network, and engine, and updates tests and database migrations accordingly.

Introduces ChainOrchestrator in place of Indexer, refactors RollupNodeManager to consume orchestrator events instead of indexer events.
Adds Synced notifications to L1Watcher and updates engine driver to handle optimistic sync via ChainOrchestrator.
Refactors configuration (ScrollRollupNodeConfig), network manager, and database migrations; adjusts tests to cover the new orchestrator flows.

Reviewed Changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
crates/indexer/src/lib.rs	Rename `Indexer` to `ChainOrchestrator` and overhaul API flows
crates/manager/src/manager/mod.rs	Replace indexer usage with `ChainOrchestrator` in node manager
crates/node/src/args.rs	Instantiate `ChainOrchestrator` in `ScrollRollupNodeConfig`
crates/watcher/src/lib.rs	Add `Synced` variant and `is_synced` flag to `L1Watcher`
crates/scroll-wire/src/protocol/proto.rs	Adjust doc comment for `NewBlock::new`
crates/node/tests/e2e.rs	Add/revise reorg and sync end-to-end tests
crates/watcher/tests/reorg.rs	Update tests to skip `Synced` notifications
crates/database/db/src/operations.rs	Extend DB ops with `L1MessageStart` and block-and-batch queries
crates/database/migration/src/migration_info.rs	Add `genesis_hash()` to migrations and insert genesis blocks
crates/network/src/manager.rs	Wire up eth-wire listener and dispatch chain-orchestrator events
crates/engine/src/driver.rs	Support `ChainImport` and `OptimisticSync` futures in engine driver

Comments suppressed due to low confidence (2)

crates/scroll-wire/src/protocol/proto.rs:33

The doc comment uses "blocks" (plural) but the constructor takes a single block; change to "block" for accuracy.

    /// Returns a [`NewBlock`] instance with the provided signature and blocks.

crates/node/tests/e2e.rs:95

The follower_can_reorg test has no assertions; either add meaningful checks or remove the empty test to maintain coverage.

async fn follower_can_reorg() -> eyre::Result<()> {

crates/watcher/tests/reorg.rs

greged93

Couple of comments and some small nits and leftover code to clean.

.github/workflows/test.yaml

crates/chain-orchestrator/src/metrics.rs

crates/chain-orchestrator/src/lib.rs

greged93 · 2025-07-23T10:40:15Z

crates/chain-orchestrator/src/lib.rs

+                        received_header_tail = header;
+                    } else {
+                        return Err(ChainOrchestratorError::MissingBlockHeader {
+                            hash: current_chain_headers.front().unwrap().parent_hash,
+                        });
+                    }
+                }
+            }
+
+            // We search the in-memory chain to see if we can reconcile the block import.
+            if let Some(pos) = current_chain_headers
+                .iter()
+                .rposition(|h| h.hash_slow() == received_header_tail.parent_hash)


I'm not sure I understand this part of the code: you fetch 50 blocks from the L2 client, starting at the received block. At the end of the fetch phase, received_header_tail = received_block - 50 and current_chain_headers contains blocks from received_block to received_block - 50. But would the above branch ever work then?

I think there is a bug here, we shouldn't be updating received_header_tail = header. Let me update this and add some test cases for deep reorgs.

updated and test added, please re-review

greged93 · 2025-07-23T15:32:21Z

crates/node/tests/sync.rs

+        signer_args: Default::default(),
+    };
+
+    // Create the chain spec for scroll dev with Euclid v2 activated and a test genesis.


nit:

Suggested change

// Create the chain spec for scroll dev with Euclid v2 activated and a test genesis.

// Create the chain spec for scroll dev with Feynman activated and a test genesis.

greged93 · 2025-07-23T15:33:45Z

crates/sequencer/tests/e2e.rs

@@ -40,7 +40,7 @@ async fn can_build_blocks() {
    reth_tracing::init_test_tracing();

    const BLOCK_BUILDING_DURATION: Duration = Duration::from_millis(0);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


greged93 · 2025-07-23T15:33:56Z

crates/sequencer/tests/e2e.rs

@@ -284,7 +281,7 @@ async fn can_build_blocks_with_finalized_l1_messages() {

    let chain_spec = SCROLL_DEV.clone();
    const BLOCK_BUILDING_DURATION: Duration = tokio::time::Duration::from_millis(0);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


greged93 · 2025-07-23T15:34:07Z

crates/sequencer/tests/e2e.rs

@@ -671,7 +674,7 @@ async fn can_build_blocks_and_exit_at_time_limit() {
    let chain_spec = SCROLL_DEV.clone();
    const MIN_TRANSACTION_GAS_COST: u64 = 21_000;
    const BLOCK_BUILDING_DURATION: Duration = Duration::from_secs(1);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


greged93 · 2025-07-23T15:35:20Z

crates/watcher/src/lib.rs

-            if self.is_synced() {
+            if self.is_synced {
                tokio::time::sleep(SLOW_SYNC_INTERVAL).await;
+            } else if self.current_block_number == self.l1_state.head {
+                // if we have synced to the head of the L1, notify the channel and set the
+                // `is_synced`` flag.
+                if let Err(L1WatcherError::SendError(_)) = self.notify(L1Notification::Synced).await
+                {
+                    tracing::warn!(target: "scroll::watcher", "L1 watcher channel closed, stopping the watcher");
+                    break;
+                }
+                self.is_synced = true;


the current logic suggests the watcher can never transition from is_synced = true -> false. Is this expected?

Good question. In the context of the RN, Synced should mean that we have synced all L1 messages required to validate messages included in unsafe L2 blocks. Given that we only include L1 messages after the corresponding L1 block has been finalized I think this should be fine provided the watcher doesn't start to lag > 2 epochs behind the safe tip then the Synced status should still remain valid. What do you think about this?

hmmm but so if we lose a provider for 12 minutes we might enter an edge case we can't exit from?

good point and given that we have had recent experiences of the L1 provider being down for longer than 12 minutes I think we should cover this case.

greged93

couple of extra comments and questions. Also I see we have a lot more unwraps in the code, are all these safe to keep?

greged93 · 2025-08-03T15:25:58Z

crates/chain-orchestrator/src/lib.rs

+        // Reverse the new chain headers to have them in the correct order.
+        received_chain_headers.reverse();


Personally think we would gain in code clarity if received_chain_headers and current_chain_headers were ordered in the same way.

greged93 · 2025-08-03T15:37:09Z

crates/chain-orchestrator/src/lib.rs

+                    let consolidated = if !*optimistic_mode.lock().await {
+                        true


we don't perform any consolidation here. Is this because in !optimistic_mode, the consolidation happens in handle_new_block?

greged93 · 2025-08-03T15:41:50Z

crates/engine/src/driver.rs

+        // Purge all pending block imports.
+        self.chain_imports.clear();


why do we purge all pending block imports?

greged93 · 2025-08-03T15:48:28Z

crates/network/src/manager.rs

+    /// Handles a new block received from the eth-wire protocol.
+    fn handle_eth_wire_block(


makes sense to have this here 👌

greged93 · 2025-08-03T15:51:08Z

crates/network/src/manager.rs

+        // Handle blocks received from the eth-wire protocol.
+        while let Some(Poll::Ready(Some(block))) =


note (not relevant for now but lets keep it in mind): at some point we might want to implement the same logic as in Reth to avoid having some components in the node monopolizing all resources.

greged93 · 2025-08-03T15:54:40Z

crates/sequencer/tests/e2e.rs

@@ -599,7 +598,7 @@ async fn can_build_blocks_and_exit_at_gas_limit() {
    let chain_spec = SCROLL_DEV.clone();
    const MIN_TRANSACTION_GAS_COST: u64 = 21_000;
    const BLOCK_BUILDING_DURATION: Duration = Duration::from_millis(250);
-    const BLOCK_GAP_TRIGGER: u64 = 100;
+    // const BLOCK_GAP_TRIGGER: u64 = 100;


greged93

reviewing l2geth code, I noticed we might be missing checks on L1 messages.

greged93 · 2025-08-07T10:17:51Z

crates/chain-orchestrator/src/lib.rs

+async fn validate_l1_messages(
+    blocks: &[ScrollBlock],
+    database: Arc<Database>,
+) -> Result<(), ChainOrchestratorError> {


I think we might be missing a couple of checks in this function:
https://github.com/scroll-tech/go-ethereum/blob/develop/core/block_validator.go#L104-L109

Post Euclid V2: check indexes are continuous, at the start of the block and start at the last seen L1 message index.

Pre Euclid V2: queue index can't decrease, check skipped L1 messages are in DB.

frisitano added 12 commits July 2, 2025 19:44

feat: chain orchestrator

c1e5cc2

Merge branch 'main' into feat/chain-orchestrator

bbe0e8f

lint

4032fa5

lint

39802e2

docs lint

c84dc0d

fix: clone the in-memory chain instead of taking it

ea3140c

Merge branch 'main' into feat/chain-orchestrator

a28a5b2

update chain consolidation and l1 message validation

fa6952c

add missing network block error

43cc2be

update fork sync / reconcilliation to use network as opossed to database

5dfc813

Merge branch 'main' into feat/chain-orchestrator

a50558d

fix merege

19458f7

frisitano mentioned this pull request Jul 15, 2025

[ChainOrchestrator] Monitor performance of recursive syncing of blocks #199

Open

frisitano added 2 commits July 16, 2025 12:56

refactor and add test cases

4578d49

cleanup

e6248f5

frisitano marked this pull request as ready for review July 16, 2025 14:10

Merge branch 'main' into feat/chain-orchestrator

626e29b

frisitano requested a review from Copilot July 16, 2025 14:13

Copilot AI reviewed Jul 16, 2025

View reviewed changes

crates/watcher/tests/reorg.rs Show resolved Hide resolved

frisitano added 2 commits July 16, 2025 22:21

rename chain orchestrator crate

7a6ebfc

cleanup

7d35185

frisitano requested a review from greged93 July 17, 2025 17:14

frisitano added 7 commits July 18, 2025 18:36

Merge branch 'main' into feat/chain-orchestrator

74fd7b8

remove expect for sequencer in rnm

fe75ed5

sync test

f3bab0d

sync test

56a0ff3

sync test

1320f14

add error handling for missing paylaod id

0fa87fe

remove networking for L1 consolidation sync test

cecbd03

greged93 mentioned this pull request Jul 22, 2025

feat: bump reth #211

Merged

greged93 reviewed Jul 23, 2025

View reviewed changes

frisitano added 7 commits July 30, 2025 01:58

Merge branch 'main' into feat/chain-orchestrator

af4bac9

improve test coverage and fix bugs

7df1a59

lint

fa51172

fix cli NetworkArgs

e86446d

add block gap to reorg integration test

9cdbdf3

Merge branch 'main' into feat/chain-orchestrator

01a10cf

make test more robust

2f851b1

greged93 reviewed Aug 3, 2025

View reviewed changes

greged93 mentioned this pull request Aug 5, 2025

L1 message validation scroll-tech/reth#243

Open

frisitano added 2 commits August 7, 2025 00:48

Merge branch 'main' into feat/chain-orchestrator

9f88b5d

merge upstream

9a3339a

greged93 reviewed Aug 7, 2025

View reviewed changes

refactor

000ade7

	// Create the chain spec for scroll dev with Euclid v2 activated and a test genesis.
	// Create the chain spec for scroll dev with Feynman activated and a test genesis.

		// Reverse the new chain headers to have them in the correct order.
		received_chain_headers.reverse();

		// Purge all pending block imports.
		self.chain_imports.clear();

		/// Handles a new block received from the eth-wire protocol.
		fn handle_eth_wire_block(

		// Handle blocks received from the eth-wire protocol.
		while let Some(Poll::Ready(Some(block))) =

feat: chain orchestrator #185

Are you sure you want to change the base?

feat: chain orchestrator #185

Uh oh!

Conversation

frisitano commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

greged93 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greged93 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greged93 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

frisitano commented Jul 2, 2025 •

edited

Loading